Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the pidfile accessible by everyone #3252

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

mcasquer
Copy link
Collaborator

@mcasquer mcasquer commented Oct 1, 2024

Creates the pidfile at an accessible location
for every user, this way the manage pidfile
warning is avoided.

Pull Request Checklist

  • implement the feature
  • write the documentation
  • extend the test coverage
  • update the specification
  • mention the version
  • include a release note

@mcasquer mcasquer marked this pull request as ready for review October 1, 2024 11:27
@mcasquer
Copy link
Collaborator Author

mcasquer commented Oct 1, 2024

To discuss here which of the added # noqa could be avoided

tmt/steps/execute/internal.py Outdated Show resolved Hide resolved
@mcasquer mcasquer force-pushed the 2877_pidfile_creation branch 3 times, most recently from 22ff571 to f5768f3 Compare October 11, 2024 10:23
@psss psss added this to the 1.38 milestone Oct 17, 2024
@mcasquer mcasquer force-pushed the 2877_pidfile_creation branch 2 times, most recently from 83819a0 to d2dafc9 Compare October 22, 2024 09:17
@happz happz added the ci | full test Pull request is ready for the full test execution label Oct 22, 2024
@mcasquer mcasquer force-pushed the 2877_pidfile_creation branch 2 times, most recently from d45af53 to 0dc6e94 Compare October 24, 2024 12:15
@thrix thrix modified the milestones: 1.38, 1.39 Oct 25, 2024
@psss psss changed the title Makes the pidfile accessible by everyone Make the pidfile accessible by everyone Oct 31, 2024
tmt/steps/execute/internal.py Outdated Show resolved Hide resolved
@mcasquer
Copy link
Collaborator Author

@happz one question, in the related issue with the patch it's used tmt -a provision -h local which IIUC it's the local host already executing tmt, in this case the code raises again permissions issues when doing the chmod

@happz
Copy link
Collaborator

happz commented Oct 31, 2024

@happz one question, in the related issue with the patch it's used tmt -a provision -h local which IIUC it's the local host already executing tmt, in this case the code raises again permissions issues when doing the chmod

Is your change here causing a regression? Or is your change here just making no difference WRT #2877?

@psss
Copy link
Collaborator

psss commented Nov 11, 2024

Summary from the chat discussion: /tests/provision/facts/guest should be adjusted to use the container image with password-less sudo that is localhost/tmt/tests/container/fedora/39/unprivileged and we will explicitly document that one of the following is needed on the guest:

  • root user (with all permissions)
  • regular user (with password-less sudo configured)

@mcasquer mcasquer force-pushed the 2877_pidfile_creation branch 2 times, most recently from fcf637b to b3a7b33 Compare November 12, 2024 06:52
@@ -40,7 +42,7 @@ rlJournalStart
fi

elif [ "$PROVISION_HOW" = "container" ]; then
provision_options="--image fedora:39"
provision_options="--image $TEST_IMAGE_PREFIX/fedora/39/unprivileged:latest"
bfu_provision_options="$provision_options --user=nobody"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--user would need to change as well, to fedora, and the main image shouldn't be the unprivileged one. Together, I think something like this should be the outcome:

provision_options="--image $TEST_IMAGE_PREFIX/fedora/39:latest"
bfu_provision_options="--image $TEST_IMAGE_PREFIX/fedora/39/unprivileged:latest --user=fedora"

@mcasquer mcasquer force-pushed the 2877_pidfile_creation branch 2 times, most recently from 6c55455 to 88a81e5 Compare November 12, 2024 10:17
@mcasquer
Copy link
Collaborator Author

It seems now there's an issue when setting the ACLs, the user doesn't have permissions

cmd:
            mkdir -p /var/tmp/tmt;
            setfacl -d -m o:rX /var/tmp/tmt
            err: setfacl: /var/tmp/tmt: Operation not permitted

Also I see some extra error about an rsync command along with the following message
fail: Failed to push workdir to the guest. This usually means that login as 'fedora' to the guest does not work.

Copy link
Collaborator

@martinhoyer martinhoyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

tmt/steps/provision/__init__.py Show resolved Hide resolved
@mcasquer
Copy link
Collaborator Author

I see now the failed tests are
/tests/core/environment-file

:: [ 10:30:11 ] :: [   FAIL   ] :: Command 'tmt run --environment STR=bad_str -rvvvddd plan --name /plan/good 2>&1             | tee output' (Expected 1, got 2)
:: [ 10:30:11 ] :: [   FAIL   ] :: File 'output' should contain 'AssertionError: assert 'bad_str' == 'O'' 

/tests/prepare/ansible

:: [ 10:54:20 ] :: [   FAIL   ] :: Command 'tmt run -i /tmp/tmp.7zaDNljweN --scratch -av provision -h container -i localhost/tmt/tests/container/alpine:latest plan -n /remote' (Expected 0, got 2)
:: [ 10:54:20 ] :: [   FAIL   ] :: File '/tmp/tmp.7zaDNljweN/log.txt' should contain 'ansible-playbook -vvv' 

Sorry I don't see how is this related with the patch... 😕 @happz @psss could you shed some light here?

@psss
Copy link
Collaborator

psss commented Nov 15, 2024

Hmmm, these seem to be caused by GitHub randomly refusing to serve the raw content:

fail: Failed to fetch remote playbook 'https://raw.githubusercontent.com/teemtee/tmt/main/tests/prepare/ansible/data/playbook.yml'.
403 Client Error: Forbidden for url: https://raw.githubusercontent.com/teemtee/tmt/22a46a4a6760517e3eadbbff0c9bebdb95442760/tests/core/env/data/vars.yaml

Sounds like they enabled some rate limiting, or we hit the limit:

Any suggestions?

@mcasquer
Copy link
Collaborator Author

I see this test case is also failing:

Command '
mkdir -p /var/tmp/tmt;
setfacl -d -m o:rX /var/tmp/tmt
' returned 1.

# stderr (1 lines)
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
setfacl: /var/tmp/tmt: Operation not permitted
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I think the same approach for the pidfile can be used?, something like:

sudo = 'sudo' if not self.facts.is_superuser else ''
...
    {sudo} setfacl -d -m o:rX {workdir_root}
    """))

what do you think? @psss @happz

@psss
Copy link
Collaborator

psss commented Nov 21, 2024

I think the same approach for the pidfile can be used?, something like:

sudo = 'sudo' if not self.facts.is_superuser else ''
...
    {sudo} setfacl -d -m o:rX {workdir_root}
    """))

what do you think? @psss @happz

Yes, I think sudo should be used here as well. This part of code is under the following condition:

if not self.facts.is_superuser and self.become

Meaning that we should probably use it here always. @happz, any thoughts?

@carlosrodfern, as this setfacl code was submitted by you, do you have any additional recommendations here?

@psss
Copy link
Collaborator

psss commented Nov 21, 2024

Anyway, this is too late for 1.39. Let's merge this as one of the first changes in 1.40 in order to give it some testing.

@psss psss modified the milestones: 1.39, 1.40 Nov 21, 2024
@carlosrodfern
Copy link
Contributor

carlosrodfern commented Nov 21, 2024

For what I can understand, when the sudo mkdir -p is run with the pid directory, which goes pass /var/tmp/tmt/, then the whole structure is created owned by root. So, when it gets to the setfacl part, it fails. I would be also surprised that the chmod ugo+rwx is enough in a umask 0027 scenario. Wouldn't it better to create /var/tmp/tmt first with the corresponding acl to avoid issues with umask 0027, and then do the nested dirs?

I may be missing something too...

Creates the pidfile at an accessible location
for every user, this way the manage pidfile
warning is avoided.

Signed-off-by: mcasquer <[email protected]>
when it doesn't exist.

Signed-off-by: mcasquer <[email protected]>
Also formats the pid directory creation command.

Signed-off-by: mcasquer <[email protected]>
This way we avoid TestingFarm to set the old values.

Signed-off-by: mcasquer <[email protected]>
in the pidfile creation. Also includes the super().setup() call to
the setup GuestSsh function.

Signed-off-by: mcasquer <[email protected]>
every time the user has no privileges

Signed-off-by: mcasquer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci | full test Pull request is ready for the full test execution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Only first user can run provision -h local. Other lack lock file permissions.
6 participants